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Abstract —Patch-based sparse representation and low-rank 
approximation for image processing attract much attention in 
recent years. The minimization of the matrix rank coupled with 
the Frobenius norm data fidelity can be solved by the hard 
thresholding filter with principle component analysis (PCA) or 
singular value decomposition (SVD). Based on this idea, we 
propose a patch-based low-rank minimization method for image 
denoising, which learns compact dictionaries from similar patches 
with PCA or SVD, and applies simple hard thresholding filters to 
shrink the representation coefficients. Compared to recent patch- 
based sparse representation methods, experiments demonstrate 
that the proposed method is not only rather rapid, but also 
effective for a variety of natural Images, especially for texture 
parts in images. 

Index Terms —Image denoising, patch-based method, low- 
rank minimization, principal component analysis, singular value 
decomposition, hard thresholding 


I. Introduction 

I MAGE denoising is a classical image processing problem, 
but it still remains very active nowadays with the massive 
and easy production of digital images. We mention below 
some important works among the vast literature which deals 
with image denoising. 

One category of denoising methods concerns transform- 
based methods, for example Q, El. The main idea is to 
calculate wavelet coefficients of images, shrink the coefficients 
and finally reconstruct images by inverse transform. These 
methods apply fixed transform dictionaries to whole images. 
However, fixed dictionaries do not generally represent whole 
images very well due to the complexity of natural images. 
Many image details are lost while being denoised. 

Another category is related to patch-based methods first 
proposed in 0 , which explores the non-local self-similarity of 
natural images. Inspired by this “patch-based” idea, the authors 
of K-SVD in and BM3D Q proposed using dictionaries to 
represent small local patches instead of whole images so that 
sparsity of coefficients can be increased, where the dictionaries 
are fixed or adaptive, and compact or overcomplete. These 
methods greatly improve the traditional methods ID, El, 
leading to very good performance. Since these works many 
similar methods have been proposed to improve the denoising 
process, such as LPG-PCA lb), ASVD Q, PLOW E), SAIST 
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ID, NCSR ns, GLIDE HIl, and WNNM However, 
many proposed methods are computationally complex. Lor 
example, K-SVD uses overcomplete dictionaries for sparse 
representation, which is time-consuming. BM3D and LPG- 
PCA iterate the denoising process twice; SAIST and WNNM 
iterate about 10 times. The computational cost is directly 
proportional to the number of iterations. 

At the same time, the low-rank matrix approximation has 
been widely studied and applied to image processing m, 
m, na. Many low-rank models have no explicit solution. 
However, the paper na proves that the nuclear norm mini¬ 
mization with the Lrobenius norm data fidelity can be solved 
by a soft thresholding filter. (See also the paper ifT^ where 
an alternative proof is given.) Lurthermore, with the help 
of Eckart-Young theorem ifTbll . the paper ifTTll demonstrates 
that the solution of the exact low-rank matrix minimization 
problem ((q norm) can be obtained by a hard thresholding 
filter. 

Inspired by the above theories, in this paper, a patch- 
based low-rank minimization (PLR) method is proposed for 
image denoising. Lirst, similar patches are stacked together 
to construct similarity matrices. Then each similarity matrix 
is denoised by minimizing the matrix rank coupled with the 
Lrobenius norm data fidelity. The minimizer can be obtained 
by a hard thresholding filter with principle component analysis 
(PCA) or singular value decomposion (SVD). The proposed 
method is rather rapid, since we use compact dictionaries 
which are more computationally efficient than over-completed 
dictionaries, and we do not iterate. Moreover, experiments 
show that the proposed method is as good as the state-of- 
the-art methods, such as K-SVD lH, BM3D 0, LPG-PCA 
0, ASVD 0, PLOW El, SAIST H), and WNNM 1121 . 

The rest of the paper is organized as follows. In Section II, 
we introduce our method. The experimental results are shown 
in Section III. Linally, this paper is concluded in Section IV. 

II. Patch-Based Low-Rank Minimization 

The noise model is; 

V = u + r), 

where u is the original image, v is the noisy one, and r/ is 
the Gaussian noise with mean 0 and standard deviation cr. The 
images m, v, 77 are with size MxN. Without loss of generality, 
we suppose that M = N. 

A. Proposed Algorithm 

Divide the noisy image v into overlapped patches of size 
d X d. Denote the set of all these patches as S = {xi : i = 


2 


,(7V-d+l)2}. 

For each patch x G S, called reference patch, consider all 
the overlapped patches contained in its n x n neighborhoocOI 
(the total number of such patches is {n — d + 1)^ patches). 
Then choose the m {m > d^) most similar patches (including 
the reference patch itself) to the reference patch among the 
{n — d + 1)^ patches. The similarity is determined by the P- 
norm distance. 

Next, for each reference patch, its similar patches are 
reshaped as vectors, and stacked together to form a matrix 
of size d"^ X m, called similarity matrix. The similarity matrix 
is denoted as = (si, S2, • • • , Sm), where columns of S, i.e. 
Si, i = 1,2, - ■ ■ ,m, are vectored similar patches. Then all the 
patches in the matrix S are denoised together using the hard 
thresholding method with the principal component (PC) basis, 
or equivalently, with the singular value decomposition (SVD) 
basis derived from the matrix S', the detailed process will be 
given afterward. For convenience, we assume that the mean 
of the patches in S, denoted by '■= ^ i 0. In 
practice, we subtract Sc from Si to form the matrix S, and 
add Sc to the final estimation si of each patch. 

Since the patches are overlapped, every pixel is finally 
estimated as the average of repeated estimates. 

The process of denoising the matrix S is shown as follows. 
Firstly, we derive adaptive basis using PCA. The PC basis 
is the set of the eigenvectors of . Write the eigenvalue 
decompositior0 

SS^ = PAP"i (1) 

with 


p= {ai,92G" = diag(TOAi,mA2,--- 


where denotes the i-th column of P and diag(ci, C 2 , • • •) 
denotes the diagonal matrix with (ci, C 2 , • • •) on the diagonal. 
The PC basis is the set of the columns of P, that is. 

The original patches Si in the similarity matrix S are 
estimated as follows; 

si^^ak{si,gi,)gk, l = l,2,--- ,m ( 2 ) 

k=l 


where 


ak 


1 if A^ > P, 
0 otherwise, 


(3) 


t being the threshold. Or equivalently, the matrix composed 
of estimated patches ([21l can be written as 


with 


S '.= {si,S2,'" ,s^) = PhiA)P-^S, 
h{A) = diag(ai, 02 , • • • , 0 ^ 2 ). 


Note that 


1 

m 


'^i{shgk)f = 

1=1 


(4) 

(5) 

( 6 ) 


^The reference patch is located at the center of the neighborhood, if the 
parities of d and n are the same; otherwise, the reference patch is located as 
neai‘ as possible to the center of the neighborhood. 

^We assume that the matrix SS^ has full rank, and it has no identical 
eigenvalues, which are generally tme in practice. 


after a simple calculation. Thus Xk can be interpreted as the 
standard deviation of the basis coefficients. 

We could also consider the singular value decomposition 
(SVD) of S: 

S = P'SQ^, (7) 


where P is chosen as the same orthogonal matrix in ([T]i, S 
is a diagonal matrix, and Q (of size m x d"^) has orthogonal 
columns such that Q^Q = I with I the identity matrix. Then 
the denoised matrix (|4|i is equal to 

S '.= PP,^(S)Q^, (8) 

where is a diagonal matrix, with the diagonal of 

obtained by the hard thresholding operator 




^kk tf Ekk ^ fv/m, 
0 otherwise. 


k = l,2,--- ,d\ 


(9) 


In fact, the equality of (HI and ® can be demonstrated as 
follows. By the equations ([T]l and (|7]l, we have A = S^, and 
P~^S = S Q^. Furthermore, by the equations (|5]) and (|9ll, 
we get h{A)E = Thus it follows that S = S. 


B. Low-Rank Minimization 

Theorem l2.1 1 stated below is an unconstrained version of the 
Eckart-Young theorem M, and comes from Theorem 2(ii) in 
El. According to Theorem 12. II it easily follows that 

S = aTgimii\\S — X\\^mPRank{X), (10) 

where the minimum is taken over all the matrices X having 
the same size as S, and || • \\f is the Frobenius norm. Hence 
the denoised matrix S is the solution of the exact low-rank 
minimization problem. 

Theorem 2.1; The following low-rank minimization prob¬ 
lem 

X = axgmimWY — X\\'^pg Rank{X) (11) 
has the solutioijl 

A = UH^{E)V^, (12) 

where Y = is the SVD of Y, and is the hard 

thresholding operator 

TT /Vt _ / if Ekk > 

\ 0 otherwise. 


C. Choice of the Threshold 

The choice of the threshold f in Q is crucial for the 
proposed algorithm. We study it by minimizing the mean 
squared error of estimated values of vectored patches si,{l = 
1, 2, • • • ,m) in a similarity matrix S. Denote 


Si = ui+'ni, 


where ui and r/; are the vectored patches of the true image 
and the noise corresponding to si respectively. 

^ Strictly speaking, if none of the singular values of Y equals with ,fjl, the 
solution is unique, which is generally true in practice. 
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By Q or (|4]i, it can be easily obtained that 

\\si - uif = ^(ofc - l)^((£/fc, Mi))^ + al{{gk,rii)f. (13) 

k=l k=l 

Assume that the PC basis { 91 , 921 ’ ’ only depends 

on the true value vectors ,Mm} and hence is 

independent of { 91 , 921 '' ’ j 'Hm}- Then 

E((9fc.’7i))^ = (14) 

Let 

m 

^1 =—Y^^9k.ui)f. (15) 

/=! 

Then by (|6]l, we obtain 

n>i) = 0l + a\ (16) 

Thus, from (fOl l. (fl4l l. and (fTSl) . it follows that 

m 

- Y ^ E “fc- 

^ k^l k^l 

After a simple calculation, the optimal value for is 

1 if 6»^ > cr^ 

0 otherwise. 

Since A| ~ by the optimal value of the threshold 

in ((SJi is R3 2cr^. In practice, we find that f = l.Scr is a good 
choice. 

III. Experimental Results 
In this section, we compare the performance of our PLR 
method with those of state-of-the-art methods, including the 
highly competitive method WNNM lEl proposed very re¬ 
cently. Standard gray images are utilized to test the perfor¬ 
mance of methods. For the simulation, the level of noise is 
supposed to be known, otherwise there are methods to estimate 
it; see e.g. Cl. For each image and each level of noise, all 
the methods are applied to the same noisy images. 

For our algorithm, the patch size is set to d = 7, the 
size of neighborhoods for selecting similar patches is set to 
n = 35, and the number of similar patches in a similarity 
matrix is chosen as m = 5cP. Image boundaries are handled 
by assuming symmetric boundary conditions. For the sake of 
computational efficiency, the moving step from one reference 
patch to its neighbors both horizontally and vertically is 
chosen as the size of patches, that is, 7. For other comparison 
algorithms, we utilize the original codes released by theirs 
authors. 

In Table |I] we compare the PSNR (Peak Signal-to-Noise 
Ratio) values of our PLR method with other methods. The 
PSNR value is defined by 

255A^ 

PSNR (v) = 201ogio ^-^dB, 

lit ; - u\\f 

where u is the original image, and v the restored one. As can 
be seen in Table |I] our method is generally better than K-S VD 
PI , LPG-PCA @ and PLOW llS), and sometimes even better 
than BM3D E). Furthermore, for the visual comparisons, our 


TABLE I 

PSNR VALUES FOR REMOVING NOISE FOR OUR PLR AND OTHER 
METHODS. Cam is the Cameramen image 


Image 

Lena 

Barbara 

Peppers 

Boats 

Bridge 

House 

Cam 

O- = 10 

K-SVD|^ 

35.50 

34.82 

34.23 

33.62 

30.91 

35.96 

33.74 

LPGPCAl^ 

35.72 

35.46 

34.05 

33.61 

30.86 

36.16 

33.69 

ASVDITJ 

35.58 

35.58 

33.55 

33.26 

27.76 

36.46 

31.62 

PLOWISI 

35.29 

34.52 

33.56 

32.94 

29.88 

36.22 

33.15 

PLR 

35.90 

35.50 

34.28 

33.76 

30.78 

36.57 

33.73 

BM3D1^ 

35.90 

35.39 

34.68 

33.88 

31.06 

36.71 

34.18 

SAIST|9| 

35.87 

35.69 

34.76 

33.87 

31.03 

36.52 

34.28 

WNNMI12I 

36.02 

35.92 

34.94 

34.05 

31.16 

36.94 

34.44 

o- = 20 

K-SVD|^ 

32.38 

31.12 

30.78 

30.37 

27.03 

33.07 

30.01 

LPGPCAl^ 

32.61 

31.69 

30.50 

30.26 

26.84 

33.10 

29.77 

ASVDITJ 

33.21 

32.96 

30.56 

31.79 

25.51 

33.53 

29.33 

PLOWJgJ 

32.70 

31.48 

30.52 

30.36 

26.56 

33.56 

29.59 

PLR 

33.03 

32.12 

30.90 

30.64 

27.20 

33.36 

30.12 

BM3Dl^ 

33.03 

32.07 

31.28 

30.85 

27.14 

33.77 

30.48 

SAISTjgj 

33.07 

32.43 

31.28 

30.78 

27.20 

33.80 

30.40 

WNNMI12f 

33.10 

32.49 

31.53 

30.98 

27.29 

34.01 

30.75 


TABLE II 

Running time in second for our PLR and other methods to 
REMOVE noise WITH IMAGES OF SIZE 256 X 256 


K-SVD 

LPG-PCA 

ASVD 

PLOW 

PLR 

SAIST 

WNNM 

210 

138 

337 

43 

2 

25 

134 


method is also good. For example, as can be seen in FiglT] 
our method preserves the texture parts in Lena and Barbara 
the best among all the methods. 

To have a clear comparison of complexities of different 
methods, we compare the average CPU time to remove noise 
with tr = 20 for the testing images of size 256 x 256: Peppers, 
House and Cameraman. All the codes are written in M-files 
and run in the platform of MATLAB R2011a on a 3.40GHz 
Intel Core i7 CPU processor. We do not include BM3D for 
comparison since the original code of BM3D contains MEX- 
files. The running time is displayed in second in Table HJ The 
comparisons clearly show that the proposed method is much 
faster than the others. 

IV. Conclusion 

In this paper, a patch-based low-rank minimization method 
for image denoising is proposed, which stacks similar patches 
into similarity matrices, and denoises each similarity matrix 
by seeking the minimize!' of the matrix rank coupled with the 
Frobenius norm data fidelity. The minimizer can be obtained 
by a hard threshoding filter with principle component basis or 
left singular vectors. The proposed method is not only rapid, 
but also effective compared to recently reported methods. 

References 

[1] D. Donoho and J. Johnstone, “Ideal spatial adaptation by wavelet 
shrinkage,” Biometrika, vol. 81, no. 3, pp. 425^55, 1994. 

[2] R. Coifman and D. Donoho, “Translation-invariant denoising,” Wavelets 
and statistics, vol. 103, pp. 125-150, 1995. 

[3] A. Buades, B. Coll, and J. Morel, “A review of image denoising 
algorithms, with a new one,” Multiscale Model. Simul., vol. 4, no. 2, 
pp. 490-530, 2005. 

[4] M. Elad and M. Aharon, “Image denoising via sparse and redundant 
representations over learned dictionaries,” IEEE Trans. Image Process., 
vol. 15, no. 12, pp. 3736-3745, 2006. 

















































4 



Fig. 1. Compare denoised images Lena and Barbara by our method and other methods for cr = 20. From left to right, the images are original images, 
noisy images, images denoised by BM3D, WNNM, and our PLR method. To make the differences clearer, the second row and the bottom row display parts 
of Lena images and Barbara images extracted from the first row and third row respectively. 
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